Image pickup system and communication system for use in video conference or the like

ABSTRACT

A video conference system includes a video camera for imaging participants in a conference, a plotting image input unit for receiving plotting data from a participant, and a synthesization unit for synthesizing an image including plotting image data being input and video data of the participant who is entering the plotting image data.

This is a division of prior application Ser. No. 08/892,464, filed Jul.14, 1997, U.S. Pat. No. 5,742,329 which is a Continuation of abandonedapplication Ser. No. 08/456,284 filed May 31, 1995, which is aContinuation of abandoned application Ser. No. 08/140,604, filed on Oct.21, 1993, each of which are herein incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image pickup system and acommunication system for use in, e.g., a video conference system forcommunicating at least image data and voice data between remote places.

2. Related Background Art

FIGS. 1A to 1C are schematic views showing a conventionally known videoconference system. FIGS. 1A, 1B, and 1C illustrate the first, thesecond, and the third states, respectively. Referring to FIGS. 1A to 1C,a plurality of participants or attendants in a conference, for example,four persons A to D in a distant place are generally denoted byreference numeral 1. A television (TV) camera 2 for imaging theattendants 1 at the conference is placed on a panhead (not shown) forpanning the TV camera 2. A camera control device 5 is so arranged as tofreely change the panning position of the TV camera 2. A moving imagecodec 3 codes a video signal obtained by the TV camera 2. Acommunication control unit 4 supplies the coded video signal from themoving image codec 3 and other control information to a local line 6. Acommunication control unit 7 receives the coded video signal and theother control information from the local line 6. A moving image codec 10decodes the coded video signal received by the communication controlunit 7 and supplies the decoded video signal to a TV monitor 12. Eachparticipant in the conference is displayed as a FIG. 13 on the TVmonitor 12.

A camera operation unit 8 supplies a control signal to the cameracontrol device 5 via the communication control unit 7, the local line 6,and the communication control unit 4, thereby panning the TV camera 2.

Of the above components, the TV camera 2, the moving image codec 3, thecommunication control unit 4, and the camera control device 5 arearranged in a remote place, and the communication control unit 7, themoving image codec 10, the TV monitor 12, and the camera operation unit8 are arranged in a self-station which communicates with the distantplace through the local line 6. In FIGS. 1A to 1C, for the sake ofsimplicity in explanation, only parts for imaging the attendants 1 areillustrated on the remote place side, and only parts for displaying theimage are illustrated on the self-station side. The system, however, ofcourse includes a voice system, a data system, and various operationsystems in addition to the above components.

The operation of the above arrangement will be described below.

Assume, as shown in FIG. 1A, that the person B of the attendants 1 atthe conference is imaged at the center of the screen by the TV camera 2.In this case, the image of the person B obtained by the TV camera 2 iscoded by the moving image codec 3 and transmitted from the communicationcontrol unit 4 to the self-station through the local line 6. Uponreceiving the image from the local line 6 via the communication controlunit 7, the self-station decodes the received image into the videosignal by using the moving image codec 10 and displays the video signalas the FIG. 13 on the TV monitor 12.

When the speaker among the attendants 1 is switched to the person D, thecamera operation unit 8 in the self-station is manipulated to transmit acontrol signal to the camera control device 5, thereby panning the TVcamera 2 via a panning means (not shown). Consequently, as shown in FIG.1B, the person D of the attendants 1 at the conference is imaged at thecenter of the screen by the TV camera 2. In this case, the image of theperson D obtained by the TV camera 2 is transmitted to the self-stationvia the moving image codec 3, the communication control unit 4, and thelocal line 6. In the self-station, the video signal is obtained from thelocal line 6 via the communication control unit 7 and the moving imagecodec 10 and displayed as the FIG. 13 on the TV monitor 12.

When the speaker among the attendants 1 at the conference is switched tothe person A, the camera operation unit 8 in the self-station ismanipulated to transmit a control signal to the camera control device 5,thereby panning the TV camera 2. As a result, as shown in FIG. 1C, theperson A of the attendants 1 is imaged at the center of the screen bythe TV camera 2. In this case, the image of the person A obtained by theTV camera 2 is transmitted to the self-station via the moving imagecodec 3, the communication control unit 4, and the local line 6. In theself-station, the video signal is obtained from the local line 6 via thecommunication control unit 7 and the moving image codec 10 and displayedas the FIG. 13 on the TV monitor 12.

As described above, the camera operation unit 8 of the self-station ismanipulated to pan the TV camera 2 via the camera control device 5, andso an image in which a speaking one of the attendants 1 at theconference is imaged at the center of the screen can be displayed on theTV monitor 12. Therefore, a speaking one of the attendants 1 in thedistant place can be selectively monitored as the FIG. 13 displayed onthe TV monitor 12 in the self-station.

The conventional video conference system with the arrangement asdescribed above has the following problems.

That is, various control operations, such as control performed by theoperation means in the remote place and control performed by theoperation means in the self-station are used to determine the panningposition of the TV camera 2, i.e., pan the TV camera 2 toward aparticular one of the attendants 1 at the conference in the distantplace. If, however, the TV camera 2 is panned to perform switchingbetween persons to be imaged among the attendants 1, this merely resultsin a change in the FIG. 13 displayed on the TV monitor 12 in theself-station. Therefore, it is difficult for participants in theconference in the self-station to accurately recognize the positionalrelationship between the attendants 1 in the remote place.

That is, although the attendants 1 at the conference in the distantplace are actually seated side by side at different positions, they aredisplayed at the fixed position of the TV monitor 12 in theself-station. This impairs the ambience of the conference and alsocauses an inconvenience that the TV camera 2 may be operated in anopposite direction in searching for a specific person by changing thepanning position of the TV camera 2 from the self-station.

To eliminate the above inconveniences, there is provided a method ofdisplaying all of the attendants 1 at the conference constantly by usinga plurality of TV cameras 2 and a plurality of TV monitors 12 in theself-station. Since, however, this largely increases the quantity ofinformation to be transmitted, the method is impractical when the numberof local lines 6 and the installation cost are taken into account.

Although not shown in FIGS. 1A to 1C, the above video conference systemfurther includes a camera for imaging originals, a still image monitorfor displaying the images obtained by that camera, and a plotting imageinput device (e.g., a digitizer) for inputting plotting images.

Each of these devices is used to perform, e.g., writing on a commonmaterial image to point out a particular portion.

More specifically, the same material image is displayed on the stillimage display monitors of terminals of individual participants in aconference, and each participant can perform writing in a given portionof the material image by using the plotting image input device of his orher terminal. This writing is superimposed on the material imagedisplayed on the still image display monitors of all the participants inthe conference.

In the conventional video conference system with this arrangement, aparticipant currently writing data is clearly known if a videoconference is held between two stations. If, however, three or morestations are attending at a conference, it is impossible to determine aperson currently writing data.

SUMMARY OF THE INVENTION

In view of the above-described background of the invention, theobjective of the present invention is to provide a video conferencesystem which is economical and can exactly determine relationshipbetween positions of participants and also exactly determine theparticipant who is currently inputting information. And also, the otherobjective of the present invention is to provide an image pickup systemused in the video conference system of the present invention.

According to one preferred embodiment of the present invention, there isdisclosed an image pickup system comprising image pickup means forimaging objects to be imaged, registering means for determining imagingdirections of said image pickup means in advance, memory means capableof storing a plurality of images corresponding to the individual imagingdirections set by said registering means, and synthesizing means forsynthesizing the stored images read out from said memory means into animage currently being obtained by said image pickup means incorrespondence with the individual imaging directions registered in saidregistering means.

According to another preferred embodiment of the present invention,there is disclosed a video conference system comprising a video camerafor imaging participants in a conference, plotting image input means,determining means for determining a participant in a conferenceinputting a plotting image by using said plotting image input means, andsynthesizing means for synthesizing a predetermined image, whichindicates input of the plotting image, into image data obtained by saidvideo camera and representing the participant in a conference performingthe plotting image input, on the basis of the determination result ofsaid determining means.

Other objects and advantages of the invention will become apparent fromthe following detailed description taken in conjunction with theappended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A to 1C are views showing the arrangement of a conventional videoconference system;

FIG. 2 is a block diagram showing the arrangement of a video conferencesystem according to the first embodiment of the present invention;

FIGS. 3A and 3B are views for explaining examples of images displayed ona TV monitor according to the first to third embodiments of the presentinvention;

FIGS. 4A to 4D are views for explaining preset of the panning positionsof attendants at a conference and preset of images;

FIG. 5 is a block diagram showing the arrangement of a video conferencesystem according to the second embodiment of the present invention;

FIG. 6 is a block diagram showing the arrangement of a video conferencesystem according to the third embodiment of the present invention;

FIG. 7 is a block diagram showing the arrangement of a video conferencesystem according to the fourth embodiment of the present invention;

FIGS. 8A and 8B are views showing examples of images displayed on a TVmonitor according to the fourth to seventh embodiments of the presentinvention;

FIGS. 9A and 9B are views showing other examples of images displayed onthe TV monitor according to the fourth to seventh embodiments of thepresent invention;

FIG. 10 is a block diagram showing the arrangement of a video conferencesystem according to the sixth embodiment of the present invention;

FIG. 11 is a block diagram showing the arrangement of a video conferencesystem according to the seventh embodiment of the present invention;

FIG. 12 is a block diagram showing the arrangement of a terminal of avideo conference system according to the eighth embodiment of thepresent invention;

FIG. 13 is a block diagram showing the arrangement of a four-stationvideo conference system according to the eighth embodiment of thepresent invention;

FIG. 14 is a view showing an example of an image displayed on a TVmonitor according to the eighth embodiment; and

FIG. 15 is a view showing an example of an image displayed on the TVmonitor when a station A is set in a plotting image input mode accordingto the eighth embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

One embodiment of a video conference system according to the presentinvention will be described below with reference to the accompanyingdrawings.

1st Embodiment

FIG. 2 is a block diagram showing the arrangement of a video conferencesystem according to the first embodiment of the present invention. Notethat the same reference numerals as in FIGS. 1A to 1C denote the sameparts in FIG. 2, and a detailed description thereof will be omitted.Referring to FIG. 2, a still image codec 9 in a remote place codes animage from a television (TV) camera 2 as a still image and transmits thecoded image to a local line 6 via a communication control unit 4. Astill image codec 11 in a self-station decodes the coded video signalreceived by a communication control unit 7 from the local line 6 andoutputs the decoded signal as a still image video signal. An imagememory 15 stores a plurality of images of the still image video signalfrom the still image codec 11. A memory controller 14 controls the imagememory 15 and supplies various sync signals. A picture-in-picturecircuit 16 synthesizes a plurality of video signals from the imagememory 15 into a moving image video signal received by the communicationcontrol unit 7 from the local line 6, thereby outputting a display videosignal. A panning position memory 18 is provided in association with thecommunication control unit 4 and stores a plurality of panning positionsof the TV camera 2.

A sub-picture 17 is fitted in an image of a FIG. 13 displayed on a TVmonitor 12. A camera operation unit 8 supplies the following signals tothe communication control unit 7 on the basis of operations performed byparticipants in a conference in the self-station: a panning operationsignal S for operating the panning position of the TV camera 2 in thedistant place, a conference participant set signal N for setting thenumber of attendants 1 at a conference, a preset command signal P forpresetting panning positions corresponding to the individual attendants1 in the remote place, and a participant select signal SEL for selectinga panning position corresponding to a given one of the attendants 1 atthe conference in the remote place.

The communication control unit 7 supplies, to the memory controller 14,a storage number command signal NS for setting the number of images tobe stored and an image write command signal WR for commanding storage ofa video signal. The memory controller 14 writes images in a numbercorresponding to the storage number command signal NS from thecommunication control unit 7 in the image memory 15 on the basis of theimage write command signal WR. The memory controller 14 also controlsthe image memory 15 to read out these stored images as a video signal bywhich they are located at predetermined positions. Furthermore, on thebasis of a video signal switch signal CH, the memory controller 14controls a timing at which the output from the picture-in-picturecircuit 16 is switched between the video signal from a moving imagecodec 10 and the video signal from the image memory 15.

The moving image codec 10 outputs a sync signal SY to the memorycontroller 14 to obtain synchronism between video signals. Thecommunication control unit 4 supplies a panning position control signalSR for setting the panning position of the TV camera 2 to the stillimage codec 9. A camera control device 5 controls the panning positionof the TV camera 2 in accordance with this panning position controlsignal SR.

Operations of the above arrangement will be described below withreference to display images shown in FIGS. 3A and 3B and preset statesshown in FIGS. 4A to 4D.

In a normal state, an image of a given one of the attendants 1 at theconference in the remote place obtained by the TV camera 2 in the remoteplace is coded by a moving image codec 3 and supplied from thecommunication control unit 4 to the local line 6. The coded video signalreceived by the communication control unit 7 from the local line 6 isdecoded by the moving image codec 10. The decoded video signal istransferred to the TV monitor 12 via the picture-in-picture circuit 16and displayed as the FIG. 13.

In order to recognize the faces, the number, and the seated positions ofthe attendants 1 in the distant place, participants in the conference inthe self-station perform panning and zooming of the TV camera 2 by usingthe camera operation unit 8, or acquire such information throughcommunications using voices. When the number of attendants 1 at theconference is recognized in this manner, the data is input from thecamera operation unit 8.

Consequently, the conference participant set signal N is supplied fromthe camera operation unit 8 to the communication control unit 7, settingthe preset number of panning positions of the TV camera 2. On the basisof this preset number, the communication control unit 7 supplies, to thememory controller 14, the storage number command signal NS for settingthe number of still images to be stored in the image memory 15. Uponreceiving the signal, the memory controller 14 keeps an image storagearea corresponding to the set number in the image memory 15.

Subsequently, when an operation is performed from the camera operationunit 8 so as to control the panning position of the TV camera 2, thecamera operation unit 8 supplies the panning operation signal S to thecommunication control unit 7. This panning operation signal S istransmitted from the communication control unit 7 to the communicationcontrol unit 4 through the local line 6. On the basis of the panningoperation signal S thus transmitted, the communication control unit 4outputs the panning position control signal SR to the camera controldevice 5. The camera control device 5 controls the TV camera 2 inaccordance with the panning position control signal SR, therebycontrolling the panning position of the TV camera 2.

In this first embodiment, the attendants 1 at the conference in theremote place are persons A to D seated side by side from the left to theright. Before the conference is started, the seated positions of theindividual persons are registered in an order from the left to theright.

In this case, as shown in FIG. 4A, participants in the conference in theself-station operate the camera operation unit 8 in the self-station toaim the TV camera 2 in the direction of the person A while monitoringthe panning position of the TV camera 2 on the TV monitor 12, such thatthe FIG. 13 of the person A is displayed at the center of the displayscreen of the TV monitor 12.

When the positioning for the person A is finished, the camera operationunit 8 is operated to perform an operation for registering the position.Consequently, the camera operation unit 8 supplies the preset commandsignal P to the communication control unit 7. Upon receiving the presetcommand signal P, the communication control unit 7 informs thecommunication control unit 4 of this reception through the local line 6.

As a result, the communication control unit 4 stores the panningposition of the TV camera 2 at that time in the panning position memory18. The communication control unit 4 also causes the still image codec 9to code the image of the person A of the attendants 1 at the conferenceobtained by the TV camera 2 and sends the coded image to the local line6. The communication control unit 7 causes the still image codec 11 todecode the still image coded signal of the person A received through thelocal line 6 and sends the decoded signal to the image memory 15.

The communication control unit 7 also outputs the image write commandsignal WR to the memory controller 14, thereby storing this still imagein the image memory 15. As a result, the image memory 15 stores theimage of the person A in the first area of the storage areas of fourimages. The video signal having only the image of the person A is outputto the picture-in-picture circuit 16. At the same time, the memorycontroller 14 supplies, to the picture-in-picture circuit 16, the videosignal switch signal CH by which the output image from the image memory15 is selected only at the timing corresponding to the image area of theperson A. Consequently, the TV monitor 12 displays an image formed byfitting the still image of the person A in an area A of the sub-picture17 superimposed on the FIG. 13 as the moving image of the person Atransmitted from the TV camera 2 via the moving image codec 3, thecommunication control unit 4, the local line 6, the communicationcontrol unit 7, and the moving image codec 10.

Subsequently, as shown in FIG. 4B, the camera operation unit 8 ismanipulated to pan the TV camera 2 in the direction of the person B.When an appropriate panning position is confirmed on the TV monitor 12,the camera operation unit 8 is operated to register the position.

Consequently, the camera operation unit 8 sends the preset commandsignal P to the communication control unit 7. Upon receiving the presetcommand signal P, the communication control unit 7 informs thecommunication control unit 4 of this reception through the local line 6.

The communication control unit 4 stores the panning position of the TVcamera 2 at this time in the panning position memory 18. Simultaneously,the communication control unit 4 causes the still image codec 9 to codethe image of the person B of the attendants 1 at the conference obtainedby the TV camera 2 and sends the coded signal to the local line 6. Thecommunication control unit 7 causes the still image codec 11 to decodethe still image coded signal of the person B received through the localline 6 and sends the decoded signal to the image memory 15.

The control communication unit 7 also outputs the image write commandsignal WR to the memory controller 14, thereby storing this still imagein the image memory 15. As a result, the image memory 15 stores theimage of the person B in the second area of the storage areas of fourimages in addition to the image of the person A.

The video signal having the images of the persons A and B is output tothe picture-in-picture circuit 16. At the same time, the memorycontroller 14 outputs, to the picture-in-picture circuit 16, the videosignal switch signal CH by which the output image from the image memory15 is selected only at the timings corresponding to the image areas ofthe persons A and B. Consequently, the TV monitor 12 displays an imageformed by fitting the still image of the person A in the area A and thestill image of the person B in an area B of the sub-picture 17superimposed on the FIG. 13 as the moving image of the person B suppliedfrom the TV camera 2 via the moving image codec 3, the communicationcontrol unit 4, the local line 6, the communication control unit 7, andthe moving image codec 10.

Likewise, as shown in FIGS. 4C and 4D, the above operation is repeatedlyperformed by setting the panning position of the TV camera 2 in thedirections of the persons C and D, thereby storing the images of thepersons C and D in the third and fourth areas of the storage areas offour images of the image memory 15, in addition to the images of thepersons A and B.

The video signal having the images of the persons A to D is output tothe picture-in-picture circuit 16. At the same time, the memorycontroller 14 outputs, to the picture-in-picture circuit 16, the videosignal switch signal CH by which the output image from the image memory15 is selected at only the timings corresponding to the image areas ofthe persons A to D. As a result, the TV monitor 12 displays an imageformed by fitting the still images of the persons A to D in the areas Ato D, respectively, of the sub-picture 17 superimposed on the FIG. 13 asthe moving image of the person supplied from the TV camera 2 via themoving image codec 3, the communication control unit 4, the local line6, the communication control unit 7, and the moving image codec 10.

In the remote place, on the other hand, the panning positions of the TVcamera 2 corresponding to the persons A to D are stored in the panningposition memory 18.

Through the above operation, the still images of the persons A to D arestored in the image memory 15 and fitted in the areas A to D of thesub-picture 17 superimposed on the original moving image by thepicture-in-picture circuit 16. Therefore, as shown in FIG. 3A, evenwhile the TV camera 2 in the distant place is imaging the person B andthe image of the person B is being displayed as the FIG. 13 on the TVmonitor 12, all the attendants 1 at the conference in the remote placecan be monitored constantly in the sub-picture 17.

Since, on the other hand, the order of the persons displayed in thesub-picture 17 corresponds to the order of the attendants 1 in theremote place, the positional relationship between the persons can alsobe known. In addition, the attendants 1 at the conference in the distantplace can be recognized better by the participants in the self-stationif the names or symbols corresponding to the individual persons aredisplayed in addition to the image of the FIG. 13 on the display screenof the TV monitor 12 or in the sub-picture 17. This can be realizedeasily by using a character generator.

In the state in which the person B is imaged by the TV camera 2 in theremote place and the moving image of the person B is displayed as theFIG. 13, as shown in FIG. 3A, if a given participant in the self-stationrequests the moving image of the person D while monitoring thesub-picture 17, an operation for selecting the person D is performed byusing the camera operation unit 8.

Consequently, the camera operation unit 8 sends the participant selectsignal SEL to the communication control unit 7, and the signal issupplied to the communication control unit 4 through the local line 6.Upon receiving the signal, the communication control unit 4 reads outthe panning position of the TV camera 2 corresponding to the person Dfrom the panning position memory 18 and supplies the readout data to thecamera control device 5. As a result, the TV camera is controlled to thepanning position for imaging the person D, thereby starting imaging theperson D. The image of the person D is transmitted and displayed as theFIG. 13 on the TV monitor 12, as shown in FIG. 3B. At the same time, thename or the symbol to be displayed is switched to that corresponding tothe person D.

If the number of attendants 1 is changed during the conference, this newnumber is reset from the camera operation unit 8, changing the areas ofthe image memory 15, and performing the preset operation again. In thiscase, whether the whole procedure is to be performed from scratch oronly an addition/delete operation is to be performed can be determinedfreely by software incorporated in the system.

In addition, the participant select signal SEL output from the cameraoperation unit 8 may be either an ID signal for selecting one particularperson directly or a scan control signal for selecting a person to theright or the left of the FIG. 13 being currently displayed.

In the above embodiment, the display areas of the sub-picture 17 arearranged side by side. However, any given arrangement can be selectedprovided that the positional relationship between the attendants 1 atthe conference in the remote place can be recognized.

2nd Embodiment

FIG. 5 is a block diagram showing the arrangement of a video conferencesystem according to the second embodiment of the present invention. Thesame reference numerals as in FIG. 2 denote the same parts in FIG. 5,and a detailed description thereof will be omitted. The above firstembodiment exemplifies the arrangement in which the panning positions ofa TV camera 2 are preset in a self-station. In the video conferencesystem shown in FIG. 5, however, a camera operation unit 22 provided ina distant place allows input of necessary information, such as aconference participant set signal N corresponding to the number ofattendants 1 at a conference, a panning operation signal S for panning aTV camera 2 to a preset position, a preset command signal P forpresetting the panning position of the TV camera 2, and a participantselect signal SEL for controlling the panning position of the TV camera2 to the preset panning position. These signals are transmitted from acommunication control unit 4 to a communication control unit 7 through alocal line 6. Therefore, preset of the panning positions of the TVcamera 2 or an operation for controlling the panning position of the TVcamera 2 after the preset can be performed from either a cameraoperation unit 8 or the camera operation unit 22. This arrangement canachieve the same effect as in the first embodiment and can also providean effect that the TV camera 2 can be operated in exactly the samemanner in both the remote place and the self-station.

3rd Embodiment

FIG. 6 is a block diagram showing the arrangement of a video conferencesystem according to the third embodiment of the present invention. Thesame reference numerals as in FIG. 5 denote the same parts in FIG. 6,and a detailed description thereof will be omitted. In the videoconference system shown in FIG. 6, a camera operation unit 22 providedin a distant place allows input of necessary information, such as aconference participant set signal N corresponding to the number ofattendants 1 at a conference, a panning operation signal S for panning aTV camera 2 to a preset position, a preset command signal P forpresetting the panning position of the TV camera 2, and a participantselect signal SEL for controlling the panning position of the TV camera2 to the preset panning position. A communication control unit 4 causesa memory controller 21 to control an image memory 20 which receives theoutput from the TV camera 2 in the remote place, generating an originalpicture of a sub-picture 17 beforehand in the remote place. This pictureis set in an image memory 15 in a self-station via a still image codec9, the communication control unit 4, a local line 6, a communicationcontrol unit 7, and a still image codec 11. This arrangement can achievethe same effect as in the above embodiments and can also provide aneffect that the preset operation need only be performed once even if thedistant place has a plurality of partner's stations.

4th Embodiment

FIG. 7 is a block diagram showing the arrangement of a video conferencesystem according to the fourth embodiment of the present invention. Thesame reference numerals as in FIG. 2 denote the same parts in FIG. 7,and a detailed description thereof will be omitted.

Referring to FIG. 7, a level conversion circuit 19 performs levelconversion such that the luminance of a particular portion of a videosignal supplied from a picture-in-picture circuit 16 to a TV monitor 12becomes different from those of other portions.

A memory controller 14' writes images in a number corresponding to astorage number command signal NS from a communication control unit 7into an image memory 15 on the basis of an image write command signalWR, and also controls the image memory 15 such that these stored imagesare read out as a video signal by which they are arranged at respectivepredetermined positions. In addition, on the basis of a video signalswitch signal CH, the memory controller 14' controls a timing at whichthe output from the picture-in-picture circuit 16 is switched between avideo signal from a moving image codec 10 and the video signal from theimage memory 15.

Furthermore, the memory controller 14' supplies, to the level conversioncircuit 19, a level conversion control signal LC by which the level of aspecific area in a sub-picture 17 of the image from thepicture-in-picture circuit 16 becomes different from those of otherareas.

Only an operation of this embodiment different from that of the firstembodiment will be described below.

The difference of this embodiment from the first embodiment is that anarea in the sub-picture 17 corresponding to a person whose moving imageis displayed on the TV monitor 12 has a normal luminance level, but theluminance levels of other areas in the sub-picture 17 are set to belower than the normal luminance level.

That is, a speaking person of participants in a conference can berecognized rapidly by emphasizing the area of the sub-picture 17corresponding to the person displayed as a moving image on the TVmonitor 12.

As shown in FIG. 8A, therefore, while a person B is imaged by a TVcamera 2 in a remote place and the image of the person B is displayed asa FIG. 13 on the TV monitor 12 as a result of the above presetoperation, attendants 1 at the conference in the remote place can bemonitored in the sub-picture 17 at any instant. It is also possible toclearly identify the area of the sub-picture 17 corresponding to theFIG. 13 because the area B of the sub-picture 17 is displayed brighterthan the other areas.

In the state in which the person B is imaged by the TV camera 2 in thedistant place and the moving image of the person B is displayed as theFIG. 13 as shown in FIG. 8A, if participants in a self-station requestthe moving image of a person D while monitoring the sub-picture 17, anoperation for selecting the person D is performed by using a cameraoperation unit 8.

Consequently, the camera operation unit 8 sends a participant selectsignal SEL to a communication control unit 7, and this signal issupplied to a communication control unit 4 through a local line 6. Uponreceiving the signal, the communication control unit 4 reads out thepanning position of the TV camera 2 corresponding to the person D from apanning position memory 18 and supplies the data to a camera controldevice 5.

As a result, the TV camera 2 is controlled to the panning position atwhich the person D is imaged, thereby starting imaging the person D. Theimage of the person D is transmitted and displayed as the FIG. 13 on theTV monitor 12 as shown in FIG. 8B. Simultaneously, the name or thesymbol to be displayed is switched to that corresponding to the personD.

At the same time, the brightness of an area D corresponding to theperson D is set to be higher than those of the other areas by the levelconversion control signal LC transmitted from the communication controlunit 4 to the level conversion circuit 19. Therefore, the person D asthe FIG. 13 being currently displayed can be clearly identified as theperson corresponding to the area D, making a clear recognition of thepositional relationship with the other areas possible.

Note that a particular area of the sub-picture is emphasized by changingthe luminance level of that area in this embodiment, but a specific areaof the sub-picture can also be emphasized by color, e.g., by displayingthe edges of the area in red.

5th Embodiment

The above fourth embodiment exemplifies the arrangement in which an areaof a sub-picture 17 corresponding to a FIG. 13 displayed on a TV monitor12 is distinguished from other areas by increasing the brightness ofthat area to be higher than those of the other areas. As shown in FIGS.9A and 9B, in contrast, the area of the sub-screen 17 corresponding tothe FIG. 13 displayed on the TV monitor 12 can be discriminated fromother areas by decreasing the brightness of that area to be lower thanthose of the other areas. This also makes it possible to clearlyrecognize the positional relationship of the FIG. 13 with the otherareas. In addition, the area of the sub-picture 17 corresponding to theFIG. 13 can also be distinguished by the color of that area rather thanthe brightness. In this case, the same effect as described above can berealized by using a hue conversion circuit in place of a levelconversion circuit 19.

6th Embodiment

FIG. 10 is a block diagram showing the arrangement of a video conferencesystem according to the sixth embodiment of the present invention. Thesame reference numerals as in FIG. 5 or 7 denote the same parts in FIG.10, and a detailed description thereof will be omitted.

The above fifth embodiment exemplifies the arrangement in which thepanning positions of a TV camera 2 are preset in a self-station. In thevideo conference system shown in FIG. 10, however, a camera operationunit 22 provided in a distant place allows input of necessaryinformation, such as a conference participant set signal N correspondingto the number of attendants 1 at a conference, a panning operationsignal S for panning the TV camera 2 to a preset position, a presetcommand signal P for presetting the panning position of the TV camera 2,and a participant select signal SEL for controlling the panning positionof the TV camera 2 to the preset panning position. These signals aretransmitted from a communication control unit 4 to a communicationcontrol unit 7 through a local line 6. Therefore, preset of the panningpositions of the TV camera 2 or an operation for controlling the panningposition of the TV camera 2 after the preset can be performed fromeither a camera operation unit 8 or the camera operation unit 22. Thisarrangement can achieve the same effect as in the above embodiments andcan also provide an effect that the TV camera 2 can be operated inexactly the same manner in both the remote place and the self-station.

7th Embodiment

FIG. 11 is a block diagram showing the arrangement of a video conferencesystem according to the seventh embodiment of the present invention. Thesame reference numerals as in FIG. 10 denote the same parts in FIG. 11,and a detailed description thereof will be omitted.

In the video conference system shown in FIG. 11, a camera operation unit22 provided in a distant place allows input of necessary information,such as a conference participant set signal N corresponding to thenumber of attendants 1 at a conference, a panning operation signal S forpanning a TV camera 2 to a preset position, a preset command signal Pfor presetting the panning position of the TV camera 2, and aparticipant select signal SEL for controlling the panning position ofthe TV camera 2 to the preset panning position. A communication controlunit 4 causes a memory controller 21 to control an image memory 20 whichreceives the output from the TV camera 2 in the remote place, generatingan original picture of a sub-picture 17 beforehand in the remote place.This picture is set in an image memory 15 in a self-station via a stillimage codec 9, the communication control unit 4, a local line 6, acommunication control unit 7, and a still image codec 11. Thisarrangement can achieve the same effect as in the above embodiments andcan also provide an effect that the preset operation need only beperformed once even if the distant place has a plurality of partner'sstations.

The system of the seventh embodiment further comprises a voice directiondetection unit 30 for detecting a speaking one of participants in aconference.

Although the voice direction detection unit 30 may detect actual voices,performing determination by using input voices from a microphone (notshown) is easier.

With this voice direction detection unit 30, it is possible to changethe imaging direction of the TV camera 2 toward a speaking personautomatically as well as to change it manually.

Switching between the manual mode and the auto mode is performed by acamera operation unit 8 or the camera operation unit 22.

A mode signal from the camera operation unit 8 or 22 is supplied to thevoice direction detection unit 30 via the communication control unit 4.In response to this input signal, the voice direction detection unit 30detects the direction in which a voice is produced.

The voice direction detection unit 30 sends the detection result to thecommunication control unit 4, and the communication control unit 4controls a camera control device 5 on the basis of the panning positioninformation stored in a panning position memory 18 and the voicedirection detection result, changing the imaging direction of the TVcamera 2. In this case, the panning position information is used inaddition to the voice direction detection result to change the imagingdirection of the camera in order to image a speaking person morereliably as an object to be imaged.

Note that in each of the above first to seventh embodiments of thepresent invention, only the arrangement for imaging attendants at aconference is illustrated on the remote place side, and only thearrangement for displaying images is illustrated on the self-stationside, for the sake of simplicity in explanation. It is, however,apparent that the self-station has the same arrangement for imagingattendants at a conference as that of the distant place, and the distantplace has the same arrangement for displaying images as that of theself-station.

Note also that in each of the above first to seventh embodiments, thevideo conference system having two stations has been described for thesake of simplicity. In the following embodiment of the presentinvention, a video conference system for holding a conference between aplurality of stations will be described.

8th Embodiment

FIG. 12 is a block diagram showing the arrangement of a terminal for avideo conference system according to a video conference system of theeighth embodiment of the present invention. Referring to FIG. 12, thisvideo conference system comprises a camera 110 for imaging participantsin a conference, a synthesization circuit 111 for superimposing apredetermined pattern on an output image from the camera 110, anoriginal image input camera 116 for inputting an image of an original114 placed on an original table 112, a plotting image input device 120for inputting a plotting image formed by an input pen 118, a movingimage display monitor 122, a still image display monitor 124, and animage control circuit 126 for selecting and synthesizing an output imagefrom the synthesization circuit 111, an output image from the camera116, and a received image, and controlling image output to the monitors122 and 124.

The system further comprises a microphone 128, a loudspeaker 130, and avoice control circuit 132 for controlling an input voice from themicrophone 128 and an output voice to the loudspeaker 130.

A communication control circuit 134 controls a connection to acommunication line, such as an ISDN line. A system control circuit 136controls the whole system in accordance with an operation of anoperation device 138 and an input plotting image from the plotting imageinput device 120. More specifically, the system control circuit 136transfers input plotting image data from the plotting image input device120 to the image control circuit 126 and, if necessary, to otherstations. The image control circuit 126 in each of the self-station andthe other stations displays the plotting image on the still imagemonitor 124. This allows each participant in a conference to performwriting in a given portion of a common material image. It is, of course,also possible to permit only a specific participant in a conference toperform writing.

The system control circuit 136 also controls the synthesization circuit111 to synthesize a designated character pattern image into the outputimage from the camera 110. The pattern image synthesized by thesynthesization circuit 111 is an image representing characters thatindicate colors, such as red, blue, and green.

In this embodiment, a specific color is assigned to a station or aterminal which is allowed to input a plotting image by using theplotting image input device 120, and the output plotting image from thestation or the terminal is displayed in that color. At the same time,the synthesization circuit 111 in that particular station superimposes acharacter pattern indicating the assigned color on an input figure imagefrom the camera 110. This superimposed image is transmitted to otherstations and displayed on the monitor of the self-station.

The operation of this embodiment will be described in detail below bytaking a video conference held by connecting four terminals as anexample. As shown in FIG. 13, terminals 200A, 200B, 200C, and 200D offour stations A, B, C, and D are connected to an inter-multipoint videoconference communication control device 204 through digital lines 202A,202B, 202C, and 202D, respectively. Each of the terminals 200A to 200Dhas the arrangement shown in FIG. 12. The internal circuits of theterminals 200A, 200B, 200C, and 200D are denoted by the same referencenumerals, as shown in FIG. 12, which are added with symbols A, B, C, andD, respectively.

Assume, for example, that the station A transmits a still image of amaterial to the stations B, C, and D and a conference is held by usingthis material. The image control circuit 126 performs compression-codingfor image information to be transmitted and expansion-decoding for thecompressed image information received. This compressing/expandingprocessing, however, does not relate directly to the operation of thisembodiment, and so a detailed description thereof will be omitted.

At the terminal 200A in the station A, a participant in the conferenceplaces a material original on an original table 112A and operates anoperation device 138A to instruct a system control circuit 136A toperform imaging and transmission of the material original. In accordancewith the instruction, the system control circuit 136A controls an imagecontrol circuit 126A and a communication control circuit 134A to executepickup, display, and transmission of a still image. That is, the imagecontrol circuit 126A supplies the output image from a camera 116A to amonitor 124A to display the image as a still image, and also suppliesthis output image to the communication control circuit 134A. Thecommunication control circuit 134A transfers the image information fromthe image control circuit 126A to the video conference communicationcontrol circuit 204 through the digital line 202A.

The video conference communication control circuit 204 transfers theimage information from the terminal 200A to the terminals 200B, 200C,and 200D. In the terminals 200B, 200C, and 200D, communication controlcircuits 134B, 134C, and 134D supply the received image information toimage control circuits 126B, 126C, and 126D, respectively. The imagecontrol circuits 126B, 126C, and 126D output the received video signalto monitors 124B, 124C, and 124D, respectively, thereby displaying theimage as a still image.

As a result, the same still image is displayed on the still imagedisplay monitors 124A to 124D of the stations A to D.

Participants in the conference in the stations A, B, C, and D are imagedby cameras 110A, 110B, 110C, and 110D, respectively, and their figureimages are intercommunicated and displayed on moving image displaymonitors 122A, 122B, 122C, and 122D of the terminals 200A, 200B, 200C,and 200D, respectively, as a four-image multi-picture display, as shownin FIG. 14. Referring to FIG. 14, FIGS. 206A, 206B, 206C, and 206Dindicate participants in the conference in the stations A, B, C, and D,respectively.

If a plurality of participants are present in each station, their imagescan be displayed in accordance with the methods described in the abovefirst to seventh embodiments. In addition, it is apparent that thenumber of pictures in the multi-picture display is not limited to fourbut may be varied in accordance with the number of stations.

Suppose the participant in the conference in the station A operates theoperation device 138A to cause the system control circuit 136A todesignate a plotting image input mode in which plotting image input froma plotting image input device 120A is enabled. In response to thisdesignation, the system control circuit 136A enables the plotting imageinput device 120A, requesting the plotting image input device 120A totransmit plotting image information (e.g., the color of a pen and thetype of a line) currently being set. The plotting image input device120A transmits the plotting image information to the system controlcircuit 136A. The system control circuit 136A causes a synthesizationcircuit 111A to generate a character pattern corresponding to colorinformation of the plotting image information received and to synthesizethis color information into the output image from the camera 110A. If,for example, the color information indicates red, the system controlcircuit 136A causes the synthesization circuit 111A to generate acharacter image, "RED."

Consequently, as shown FIG. 15, the character image "RED" issuperimposed on the FIG. 206A of the attendant in the station A, whichis displayed on each of the monitors 122B, 122C, and 122D in thestations B, C, and D, respectively.

When a plotting image is input from the plotting image input device120A, the system control circuit 136A transfers the plotting image datato the image control circuit 126A. The image control circuit 126Adisplays the plotting image on the still image of the material displayedon the still image display monitor 124A, and at the same time transfersthe plotting image data to the communication control circuit 134A. Thecommunication control circuit 134A transfers the plotting image data tothe video conference communication control device 204 through thedigital line 202A. The video conference communication control device 204transfers the plotting image data to the terminals 200B, 200C, and 200Dthrough the digital lines 202B, 202C, and 202D, respectively.

In the terminals 200B, 200C, and 200D, the communication controlcircuits 134B, 134C, and 134D supply the received plotting image data tothe image control circuits 126B, 126C, and 126D, respectively. The imagecontrol circuits 126B, 126C, and 126D display the plotting image basedon the received plotting image data on the still image of the materialdisplayed on the still image display monitors 124B, 124C, and 124D,respectively.

As a result, the content that the participant in the conference in thestation A has written by using the plotting image input device 120A issuperimposed on the material images displayed on the still image displaymonitors 124A, 124B, 124C, and 124D in the stations A, B, C, and D,respectively.

While the participant in the conference in the station A is inputting aplotting image by using the plotting image input device 120A, the systemcontrol circuit 136A causes the synthesization circuit 111A to flickerthe pattern image currently being synthesized. This consequentlyflickers the character image "RED" superimposed on the FIG. 206Adisplayed on each of the moving image monitors 122A, 122B, 122C, and122D of the stations A, B, C, and D, respectively, shown in FIG. 15.Therefore, the participants in the conference in the stations B, C, andD can recognize that the participant in the station A is inputting aplotting image.

When the participant in the conference in the station A operates theoperation device 138A to designate the system control circuit 136A toset a plotting image input disable mode, the system control circuit 136Ainhibits the synthesization circuit 111A from synthesizing a pattern.That is, the synthesization circuit 111A directly outputs the outputfrom the camera 110A. As a result, the content displayed on each of themoving image display monitors 122A, 122B, 122C, and 122D of theindividual stations returns from that shown in FIG. 15 to that shown inFIG. 14.

In the above embodiment, the synthesization circuit 111 is operated whenthe plotting image input mode is set. However, the synthesizationcircuit 111 can also be operated when input of a plotting image isstarted. In addition, the image synthesized into the output image fromthe camera 110 by the synthesization circuit 111 may be a characterpattern image "PLOTTING" or "WRITING" generated in the correspondingcolor.

If the plotting image input mode is set in a plurality of stations, thesame color may be designated by these stations in some cases. In such acase, color adjustment must be performed to set different colors betweenthe stations. As an example, priority is given to color designationperformed by a terminal that has set the plotting image input modeearlier, and a terminal which is going to set the plotting image inputmode later checks colors used in other stations. If the same color isalready used by another station, a participant in the conference in theself-station is prompted or forced to use another color different fromthe color already used. A pattern image in the selected color issynthesized by the synthesization circuit 111 in that station.

As can be readily understood from the above description, according tothe eighth embodiment of the present invention, a participant inputtinga plotting image can be clearly known even if three or more persons aretaking part in a conference. This greatly improves the utility of thesystem.

The present invention can be carried out in various other forms withoutdeparting from the spirit and the scope of the present invention.

For example, it is obvious that the eighth embodiment is applicable tothe first to seventh embodiments, and, conversely, the first to seventhembodiments can be applied to the eighth embodiment.

One practical example is that although the screen of the monitor in theeighth embodiment is divided into portions corresponding to the numberof stations taking part in a video conference as described above withreference to FIG. 14, participants in individual stations can bedisplayed as shown in FIGS. 3A and 3B, FIGS. 8A and 8B or FIGS. 9A and9B.

In addition, when the eighth embodiment is applied to the first toseventh embodiments to hold a video conference between a plurality ofstations, a station select switch for designating a station to beselected need only be provided in the camera operation unit 8 so thatonly a station designated by the station select switch can becontrolled.

In other words, the foregoing description of embodiments has been givenfor illustrative purposes only and not to be construed as imposing anylimitation in every respect.

The scope of the invention is, therefore, to be determined solely by thefollowing claims and not limited by the text of the specifications andalterations made within a scope equivalent to the scope of the claimsfall within the true spirit and scope of the invention.

What is claimed is:
 1. A video conference system comprising;a) a videocamera for imaging participants in a conference; b) plotting image inputmeans; c) determining means for determining a participant in aconference inputting a plotting image by using said plotting image inputmeans; and d) synthesizing means for synthesizing a predetermined image,which indicates input of the plotting image, into image data obtained bysaid video camera and representing the participant in a conferenceperforming the plotting image input, on the basis of the determinationresult of said determining means.
 2. A system according to claim 1,further comprising display means for displaying the image datasynthesized by said synthesizing means.
 3. A system according to claim2, wherein said synthesizing means performs synthesization such thatwhile the plotting image input is being performed by said plotting imageinput means, the predetermined image data is turned flickered on saiddisplay means.
 4. A system according to claim 3, further comprisingtransmitting means for transmitting the image data synthesized by saidsynthesizing means.
 5. A video conference system comprising:a videocamera for picking up participants to be imaged in a conference;registering means for registering a plurality of position informationindicating directions of positions of the participants in theconference; memory means for storing a plurality of image datacorresponding respectively to said plurality of position informationregistered by said registering means, said plurality of image data beingpicked up by said video camera: synthesizing means for synthesizing theimage data read out from said memory means on image data currently beingpicked up by said video camera with a predetermined relationcorresponding to the respective position information registered in saidregistering means: plotting image input means; and determining means fordetermining a participant in a conference inputting a plotting image byusing said plotting image input means, wherein said synthesizing meanssynthesizes a predetermined image, which indicates input of the plottingimage, into image data obtained by said video camera and representingthe participant in a conference performing the plotting image input, onthe basis of the determination result of said determining means.